# Model Compression

WhisperKit is a tool for compressing and optimizing automatic speech recognition (ASR) models. It allows for model compression and optimization, providing detailed performance evaluation data. WhisperKit also offers quality assurance certifications for different datasets and model formats, and supports local reproducibility of test results.

SliceGPT is a new post-training sparsity approach that reduces the network's embedding dimension by replacing each weight matrix with a smaller (dense) matrix. Through extensive experiments, we demonstrate that SliceGPT can remove up to 25% of the model parameters (including embeddings) from LLAMA2-70B, OPT 66B, and Phi-2 models while maintaining 99%, 99%, and 90% of the zero-shot task performance, respectively. Our sliced models run on fewer GPUs and execute faster without any additional code optimizations: on a 24GB consumer-grade GPU, we reduce the total inference computation of LLAMA2-70B to 64% of the dense model; on a 40GB A100 GPU, we reduce it to 66%. We provide a new insight into the computational invariance in transformer networks, which makes SliceGPT possible. We hope it can inspire and promote new avenues for reducing memory and computational requirements of pre-trained models in the future.

The ControlNet-HandRefiner-pruned model is a fp16 version resulting from pruning and compression of the HandRefiner model, enabling faster hand image restoration. This model employs a diffusion model for conditional image completion, allowing for precise restoration of missing or malformed parts in hand images. The model boasts a high compression rate and fast inference speed, making it ideal for high-quality hand image restoration in resource-constrained environments.